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ABSTRACT:  This  paper  describes  the  creation  of  the  Distributed  Mission  Training 
Research  Network  (DMT-RNet),  the  rationale  behind  its  creation,  and  its  capabilities. 
Information  is  provided  about  the  various  simulation  platfonns  included  in  the  network 
and  how  they  have  become  integrated  to  enable  a  common  battlespace,  including 
platforms  developed  specifically  for  use  in  the  DMT-RNet  and  platforms  that  were 
developed  independently.  The  latest  interoperability  tests  and  linkages  created  are 
described.  Fidelity  issues,  agent  technology,  and  training  implications  are  discussed. 

INTRODUCTION 

Extensive  and  high-quality  training  is  central  to  the  maintenance  of  United 
States  military  dominance.  The  training,  realistic  and  extensive,  includes  many  sessions 
of  highly  monitored,  live-fire  exercises  necessary  to  refine  combat  experience  and  skill. 
In  the  Air  Force,  these  live-fire  exercises  include  highly  renowned  mission  rehearsal 
training  such  as  Red  Flag  and  Close  Air  Support  Exercises. 

However,  live-fire  sessions  also  have  inherent  disadvantages.  These  include 
significant  limitations  with  regard  to  risk,  complexity,  and  uncertainty  in  mission 
scenarios.  Limitations  are  primarily  driven  by  safety  issues,  for  training  participants  and 
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inhabitants  in  and  around  training  exercise  areas.  Environmental  issues  (e.g.,  pollution 
from  emissions,  noise,  hazardous  materials)  also  restrict  the  use  of  live-fire  exercises. 
Finally,  high  cost  is  also  a  significant  limiting  factor. 

There  are  approximately  400,000  US  Air  Force  military  personnel.  Most  personnel 
go  through  a  month  or  more  of  training  per  year,  every  year  of  their  30-year  careers.  One 
should  also  consider  nearly  the  same  numbers  of  personnel  from  the  Navy,  Army,  and  the 
Marines  who  also  require  training.  If  one  also  includes  all  the  government  civilians  who 
are  equally  required  to  go  through  training,  the  costs  are  staggering. 

Today’s  international  climate  of  dispersed  operations  in  a  multitude  of  differing 
battlefield  situations  has  brought  distributed  training  to  the  forefront  of  making  training 
better  suited  to  warrior  needs.  Each  battlefield  situation  is  distinct  and  the  technology, 
sociology,  and  psychology  of  war  rapidly  changes.  What  is  the  most  efficient,  effective, 
and  least  costly  way  to  train  our  warriors? 

Training  as  a  continuous  cycle  is  quite  costly.  Warriors  are  dispersed  globally  and 
cannot  simply  leave  in  the  middle  of  operations  to  go  to  the  United  States  for 
schoolhouse  (i.e.,  residing  at  a  specific  location)  training.  In  addition,  the  mass  of  travel 
associated  with  bringing  personnel  stateside  for  training  would  result  in  exorbitant  cost. 
We  must  be  able  to  train  personnel  from  the  operational  field  and  at  the  same  time  train 
stateside  personnel  who  have  yet  to  go  into  peacekeeping  or  battlefield  operations. 

Reserves  and  National  Guard  personnel  are  unable  to  train  in  an  active  duty 
schoolhouse  for  a  prolonged  amount  of  time  because  they  are  not  normally  on  fulltime 
active  duty  status,  yet  they  are  required  to  be  as  fully  capable  as  their  active  duty 
counterparts.  Lately,  reservist  and  guard  personnel  are  being  deployed  into  battlefield 
operations  in  ever  increasing  numbers.  According  to  Newsmax.com  (2002),  "About 
50,000  U.S.  military  personnel  are  in  the  region  fighting  in  Operation  Enduring  Freedom, 
the  Pentagon  said.  There  are  three  aircraft  carriers  and  numerous  smaller  ships  in  the 
waters  south  of  Afghanistan.  More  than  40,000  reservists  have  been  activated,  most  for 
homeland  defense  duties,  but  some  for  combat  inside  Afghanistan's  borders." 

It  would  cost  approximately  $83.4  million  in  per  diem  costs  to  ship  50,000 
personnel  stateside  for  two  weeks  of  training  assuming  they  travel  to  multiple  training 
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bases  across  the  country  for  specialty  training.  This  does  not  include  the  costs  of  multiple 
aircraft  transports,  fuel,  maintenance,  and  scheduled  crew  downtime. 

Personnel  assigned  in  the  United  States  normally  travel  from  their  base  to  their 
training  site  by  commercial  aircraft.  The  approximate  cost  of  $83.4  million  in  per  diem 
costs  for  50,000  personnel  stays  the  same  but  now  the  costs  of  airfare  are  added.  If  a 
typical  round-trip  flight  costs  approximately  $400,  multiplied  by  50,000  personnel,  the 
airfare  costs  equate  to  $20  million.  Now,  add  the  per  diem  costs  to  the  airfare  costs  and 
you  get  $103.4  million  for  50,000  personnel  for  two  weeks  of  training. 

For  several  years  now,  the  Air  Force  has  focused  many  resources  into  the 
development  of  Distributed  Mission  Training  (DMT)  to  overcome  these  limitations. 
The  purpose  of  DMT  is  to  allow  people  who  are  geographically  dispersed  to  still  train 
together  using  networks  of  highly  realistic  equipment-based  simulators.  These  simulators 
enable  complex  and  challenging  battlespace  scenarios  that  are  not  restricted  to  particular 
geographic  areas,  tactics,  maneuvers,  or  limitations  in  assets/manpower.  Highly 
experienced  tacticians  deployed  worldwide  can  brief,  mission  rehearse,  and  debrief  with 
novice  trainees  located  at  various  sites.  Advantages  range  far  beyond  cost  savings. 

The  cost  savings  associated  with  DMT  are  indeed  significant.  A  person  in  San 
Antonio  can  train  one  day  with  someone  who  is  in  Arizona,  and  the  next  day  with 
someone  in  Florida.  The  benefits  of  training  in  teams  are  retained,  while  eliminating 
much  of  the  traveling  that  is  necessary  to  do  so.  These  savings  become  obvious  when 
considering  the  travel  costs  of  flying  people  around  the  country 

For  examples  of  DMT  technologies  that  have  been  well  developed  and  are  currently 
paying  back  their  invested  money  in  the  form  of  these  savings,  see  Crane  and 
Guckenberger  (2000);  Crane,  Schiflett,  and  Oser  (2000);  Crane  (1999);  and  Crane  and 
Kang  (1999). 

DMT-RNet:  THE  NEXT  STEP 

So  far,  DMT  technologies  have  focused  on  high-technology  simulations 
networked  for  geographically  dispersed  personnel  for  the  purpose  of  mission  rehearsal. 
This  technology  continues  to  be  developed  as  its  role  in  training  becomes  more  central. 
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Air  Force  technology,  originally  aimed  at  reductions  of  physiological  limitations  such  as 
g-tolerance,  is  now  focused  on  enhancement  of  cognitive  functions  such  as  situation 
awareness,  problem  solving,  and  decision  making.  The  development  and  increased  use 
of  uninhabited  reconnaissance  and  intelligence  gathering  aircraft  underlines  the  growing 
and  central  role  of  the  human  as  leader  and  decision  maker  as  opposed  to  emphasis  on 
physiological  and  psychomotor  skills  associated  with  piloting  in  air-to-air  combat 
missions. 

However,  while  the  use  and  importance  of  DMT-based  training  increases,  we  must 
admit  to  significant  limitations  in  our  understanding  of  skill  acquisition  and  training 
effectiveness  in  large-scale,  multi-team  training  scenarios.  DMT  is  a  new  approach  in 
training  and  thus  many  questions  arise:  How  many  live  trainees  should  the  training  target 
in  a  session?  How  can  training  requirements  be  generated  and  met  for  multiple  positions 
within  a  single  session?  How  do  we  ensure  the  necessary  levels  of  control?  What  is  the 
best  ratio  of  live  trainees,  live  confederates,  and  agent-based  constructive  forces?  How 
can  one  train  and  assess  adaptive  problem  solving  in  controlled  circumstances?  How 
shall  performance  be  assessed  and  displayed?  What  are  the  optimal  timing,  format,  and 
distribution  of  performance  feedback? 

At  Brooks  City  Base  in  San  Antonio,  the  Infonnation  Systems  Training  Branch  of 
the  Warfighter  Training  Research  Division  is  taking  the  next  leap  forward  in  developing 
the  Distributed  Mission  Training  Research  Network  (Barnes,  Elliott,  Tessier,  &  Petrov, 
2002).  While  other  DMT  technologies  have  focused  on  classified  training 
infrastructures,  DMT-RNet  focuses  on  training  optimization  using  available  unclassified 
infrastructures  allowing  controlled  and  multidiscipline  university-based  investigations  of 
multi-team  training  systems.  DMT-RNet  establishes  the  first  unclassified  military 
network  of  computer-based  synthetic  task  research  platforms  used  by  a  consortium  of 
government,  universities,  and  small  businesses.  Each  platform  is  an  analogue  of  the 
cognitive  demands  of  a  military  operational  perfonnance  domain,  such  that  naive  trainees 
can  be  quickly  trained  in  the  overall  responsibilities  and  decision  making  of  a  particular 
operational  role,  e.g.,  a  command  and  control  (C2)  weapons  controller,  or  an  Uninhabited 
Aerial  Vehicle  (UAV)  operator.  These  platforms  were  developed  based  on  cognitive  task 
analyses  of  selected  Air  Force  operations.  In  addition,  they  were  engineered  to  enable 
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fine-tuned  manipulations  of  display  features,  infonnation  flow,  and  scenario  events. 
Performance  data  are  collected  and  displayed  online  and  in  extensive  data  output  files. 
As  these  data  are  analyzed,  research  findings  will  be  applied  to  the  advancement  of  DMT 
technology. 

FIDELITY  ISSUES 

DMT  enables  highly  realistic  mission  rehearsal  based  on  networks  of  high-fidelity 
simulations  that  immerse  personnel  in  virtual  battlespace  scenarios.  These  simulators  are 
highly  realistic,  in  terms  of  equipment  characteristics  and  procedures  used  in  an 
operational  setting.  They  look  and  function  just  like  the  “real  thing.”  In  fact,  some  of 
these  systems  are  fully  functional  as  operating  USAF  systems — the  main  difference  being 
the  added  capacity  to  connect  and  operate  within  a  simulation  network.  Because  these 
simulations  strive  for  maximum  realism,  they  must  run  in  classified  mode  within  its  own 
dedicated  network,  thus  restricting  data  analysis  and  publication.  Further,  systems  such 
as  these  are  extremely  expensive,  difficult  to  deploy  and  utilize  in  field  settings,  and  less 
available  for  use,  relative  to  PC-based  systems. 

Van  den  Bosch  and  Riemersma  (2003,  in  this  volume),  elaborate  further  on  the 
problems  of  high  physical  fidelity  systems.  The  goal  of  operational  missions  is  to 
minimize  failure,  while  the  goal  of  training  is  to  learn  and  improve  knowledge  and  skills. 
These  goals  are  often  incongruent.  Operational  environments  are  typically  complex, 
dynamic,  and  unpredictable,  which,  conflicts  with  the  fundamental  requirement  of 
training  and  measurement  of  having  control  over  task  content,  so  that  it  is  known  which 
events  will  occur  at  a  specified  point  in  the  scenario  (Fowlkes,  Dwyer,  Oser  &  Salas, 
1998).  Thus,  the  “train  as  you  fight”  philosophy  is  in  conflict  with  the  didactic  principle 
of  control.  Control  is  necessary  to  make  sure  that  the  content  and  complexity  of  scenarios 
is  in  accordance  with  the  phase  of  training. 

Fowlkes  et  al.  (1998)  argue  that  the  factors  that  make  wargames  exciting  (complex, 
dynamic,  and  unpredictable)  are  the  very  factors  that  make  them  troublesome  from  a 
training  standpoint.  Control  of  task  content  is  a  fundamental  requirement  of  training  in 
order  for  it  to  be  known  what  is  being  measured.  This  requirement  is  clearly  at  odds  with 
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the  intentionally  dynamic  nature  of  war  fighting  that  is  simulated  in  war  games  (van  den 
Bosch  &  Riemersma,  2003). 

In  contrast  to  many  high  physical  fidelity  systems,  DMT-RNet  systems  enable 
systematic  investigations  in  unclassified  mode  and  establish  the  infrastructure  to  conduct 
multi-level  investigations  of  operational  perfonnance  using  less  costly  PC-based  systems. 
The  DMT-RNet  collaborative  research  program  leverages  emerging  Internet-2  (1-2) 
capabilities  to  connect  distributed  PC-based  simulation  systems  and  create  complex 
environments  for  multi-operator  training  and  performance  research.  These  systems  can 
be  readily  deployed  to  operational  field  settings  and  enable  cost-effective  distributed 
training  wherever  Internet  access  is  available. 

DMT-RNet  systems  will  not  be  total  replications  of  operational  systems  (Bames, 
Elliott,  &  Entin,  2001).  Instead,  these  synthetic  team  task  systems  will  capture  the 
cognitive  and  task  demands  of  most  interest  to  trainers  and  researchers  (Elliott, 
Dalrymple,  Regian,  &  Schiflett,  2001).  For  example,  a  PC-based  system  may  simplify 
the  “button  pressing”  procedures  required  in  an  actual  operational  system  and  instead 
focus  on  display  characteristics,  decision  making  processes,  tactics,  strategies,  and/or 
teamwork  functions.  Convincing  arguments  have  been  made  for  the  relevance  of  systems 
that  are  based  on  psychological  fidelity  and  the  absolute  need  for  internal  validity  for  the 
advancement  of  scientific  knowledge  (Berkowitz  &  Donnerstein,  1982;  Cook  & 
Campbell,  1979;  Mook,  1983;  Kozlowski  &  DeShon,  this  volume).  The  research 
platforms  used  in  the  initial  phases  of  the  DMT-RNet  program  were  developed  to 
represent  the  underlying  cognitive  and  decision  making  task  demands  of  Airborne 
Warning  and  Control  System  (AWACS)  Weapons  Director  (WD)  teams,  based  on 
multiple  investigations  of  cognitive  and  functional  aspects  of  this  performance  domain 
(Coovert,  et  ah,  1999). 

Another  benefit  of  a  synthetic  task  environment  is  the  variety  of  subjects  that  can 
participate  in  the  research.  High-fidelity  simulators  are  so  close  to  reality  that  they 
necessitate  expert  operators  as  subjects.  There  is  a  shortage  of  personnel  in  the  Air  Force 
in  general,  and  particularly  for  high-expertise  tactical  operators.  The  operational  Air 
Force  simply  can’t  spare  these  people  for  all  of  its  research  issues,  regardless  of  potential 
payoff  from  these  investigations.  Systems  such  as  the  Distributed  Dynamic  Decision- 
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making  Network  (DDD-Net)  and  the  Agent-Enabled  Decision  Guide  Environment 
(AEDGE)  (discussed  in  this  chapter)  can  use  naive  subjects  and  train  them  quickly  to  use 
the  system,  thus  allowing  skill  acquisition  to  focus  on  underlying  tactics,  strategies, 
teamwork,  or  other  training/research  objectives  while  limiting  the  commitment  of 
operational  personnel  and  funds  for  the  Air  Force.  They  also  allow  researchers  to  study 
knowledge-based  skill  acquisition  processes  and  interventions  (e.g.,  training  content  and 
delivery,  distribution  and  display  of  information,  and  coaching  and  decision  support 
capabilities). 

The  issue  of  knowledge  acquisition  is  important  when  working  with  training 
objectives  that  are  remotely  administered  and  scored,  for  example,  Advanced  Distributed 
Learning  (ADL).  To  achieve  optimal  content  and  delivery,  we  must  use  cognitive 
principles  to  identify  specific  knowledge  structures  required  to  coordinate,  interpret,  and 
predict  the  activities  of  others.  This  is  fundamental  to  measure  and  enhance  individual 
and  team  situational  awareness.  We  must  measure,  from  a  functional  and  cognitive 
perspective,  how  often  people  should  update  their  knowledge,  by  what  method,  and  how 
it  should  be  shared  in  a  team. 

DMT-RNet  Platforms:  DDD-Net 

Barnes,  et  al.  (2001)  discuss  the  initial  phase  of  the  DMT-RNet  project  that  utilized 
the  (DDD)  team-in-the-loop  simulation  environment  (Hess,  MacMillan,  Elliott,  & 
Schiflett,  1999;  Kleinman  &  Serfaty,  1989).  The  DMT-RNet  program  developed  an 
internet-based  version  of  the  DDD,  the  DDD-Net,  which  allows  players  in  distributed 
locations  to  connect  and  perform  a  distributed  mission  in  real  time.  The  DDD-Net  is  an 
internet-ready  version  of  a  Linux-based  collaborative  gaming  space  that  connects  players 
to  each  other  and  to  others,  such  as  observers,  confederates,  trainers,  or  researchers  (see 
MacMillan,  et  al.,  (1998). 

In  the  DDD-Net,  observers  at  any  location  in  the  network  are  able  to  observe  the 
scenario  play  in  real  time.  They  can  view  the  screen  display  and  electronic 
communications  of  any  player  and  communicate  to  one  another  via  email  or  voice.  In 
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addition,  the  DDD-Net  can  connect  players  to  one  another  for  interactive  mission 
planning,  debriefings,  and  after-action  reviews. 

DDD  simulations  in  general  are  based  on  broad  C2  functions  and  have  been 
demonstrated  to  elicit  important  team-oriented  cognitive  processes  such  as 
communication  and  coordination,  resource  allocation  and  sharing,  and  decision  making. 
For  this  initial  effort,  the  DDD  software  and  scenarios  were  developed  as  analogues  to 
USAF  operational  performance  domains. 

Specifically,  this  version  of  the  DDD-Net  was  developed  to  represent  the  underlying 
cognitive  and  decision-making  task  demands  of  AWACS  WD  teams,  based  on  multiple 
investigations  of  cognitive  and  functional  aspects  of  this  perfonnance  domain  (Coovert, 
et  ah,  2001).  Further  development  resulted  in  a  scenario  that  emulates  three  military  C2 
teams:  the  USAF  AWACS  team,  another  USAF  ground-based  C2  team,  and  a  third  Navy 
airborne  C2  team. 

Other  platfonns,  also  based  on  USAF  operations,  will  be  utilized  and  networked  into 
DMT-RNet  in  future  efforts.  These  additional  systems  have  also  been  developed  within 
this  program,  and  each  provides  an  integrated,  internet-enabled,  collaborative  training 
space  that  supports  three  integrated  capabilities:  distributed  team  performance,  distributed 
assessment  and  distributed  training  (Elliott,  Chaiken,  Stoyen,  Petrov,  &  Riddle,  2000). 

DMT-RNet  Platforms:  AEDGE™  1 

For  example,  the  AWACS-AEDGE™  is  another  internet-ready  platform  that 
emulates  AWACS  WD  functions,  but  uses  a  java-based  architecture  composed  of  a 
unique  federation  of  intelligent  agents.  These  agents  generate  and  execute  scenario 
function,  data  collection,  decision  support,  and  emulate  WD  roles  within  the  scenario. 
This  platfonn  allows  participants  to  play  with  or  against  agent-based,  simulated  friendly 
and/or  enemy  forces.  It  can  also  execute  scenarios  in  all-agent  mode,  providing  what-if 
analyses  of  different  scenario  characteristics  (e.g.,  number  and  location  of  assets  and 
targets).  These  advanced  agent-based  capabilities  will  provide  further  improvements  to 
internet-based  training  and  research.  It  was  developed  specifically  for  the  DMT-RNet 


1  AEDGE,  AWACS-AEDGE,  and  21CSI  are  registered  trademarks  of  21st  Century  Systems  Inc. 


(Bames,  Elliott,  et  al,  2002).  The  AWACS-AEDGE,  built  using  21st  Century  Systems 
Inc.’s  AEDGE™  infrastructure,  is  a  distributed,  real-time  team  decision  support 
environment  comprised  of  simulators,  entity  framework,  intelligent  agents,  and  user 
interfaces.  The  environment  supports  a  wide  variety  of  air,  sea  (surface  and  sub-surface), 
and  ground  assets  in  a  combat  environment  (Chiara  &  Stoyen,  1997),  primarily  based  on 
the  roles  and  responsibilities  of  AWACS  WD  team  members.  The  environment  has  been 
tested  with  an  excess  of  200  physical  entities  (planes,  ships,  surface-to-air  (SAM)  sites, 
etc.)  operating  with  realistic  yet  non-classified  performance  characteristics  in  an 
interactive  environment  in  which  real-time  decision  support  is  available  to  each  WD. 

The  behavior  and  decision 
making  of  all  hostile  and  friendly 
entities  not  controlled  by  humans  is 
directed  by  agent-based  technology.  If 
a  human  decides  to  “log  in”  as  a 
particular  entity,  he/she  may  choose  to 
view  recommendations  generated  by 
the  agent  for  that  entity.  Even  if  the 
human  operator  chooses  not  to  view 
recommendations,  the  agent 
recommendations  are  still  logged  by 
the  computer.  This  enables  direct  comparison  of  human  to  agent  decision  making.  We 
expect  these  capabilities  will  facilitate  skill  acquisition,  decision  making,  skills 
assessment,  and  human/team  performance  modeling. 

AEDGE  agent  capabilities  enable  more  detailed  and  innovative  approaches  to 
measurement  and  modeling  of  individual  and  team  workload,  communication  and 
decision  making.  Tracking  the  number  and  type  of  recommendations  generated  by  the 
agent  at  any  given  time  contributes  toward  new  ways  of  conceptualizing  and  representing 
cognitive  workload  of  individuals  and  teams.  Agent-based  recommendations  may  also 
serve  as  a  standardized  benchmark  by  which  human  tactics  and  decisions  can  be 
compared.  In  addition,  the  AEDGE  platfonn  can  operate  through  speech  -  operators  can 
speak  to  the  system  using  predefined  jargon,  request  tasks  be  perfonned  or  information 
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provided/transferred,  and  the  agents  will  respond  verbally  to  the  speech-driven  requests, 
using  voice  generation  technology.  All  agent  communications  with  each  other,  as  well  as 
to  humans,  are  transcribed,  logged  to  data  output  fdes,  and  are  available  online. 

The  AWACS-AEDGE  was  conceived  through  cognitive  and  functional  analysis 
of  team  member  roles,  responsibilities,  and  decision  making  (Dalrymple,  1991),  to 
optimize  generalizability  of  results  to  operational  settings.  Systematic  descriptions  of 


Figure  1:  AEDGE  Screenshot 

AWACS  roles,  responsibilities,  requirements,  interdependencies,  tactics,  strategies,  and 
task  demands  were  collected  from  subject-matter  experts,  cognitive  task  analyses  (Fahey, 
Rowe,  Dunlap,  &  DeBoom,  1997;  MacMillan  et  ah,  1998)  and  focal-group  interviews 
(Elliott  et  ah,  1999;  Elliott  et  ah,  2001).  These  data  were  examined  to  identify  decision 
events,  which  were  generic  to  performance,  regardless  of  mission  scenario,  and  likely  to 
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bottleneck  under  high  tempo  situations  and  sustained  operations  (see  Elliott,  et  al.  this 
volume). 

AEDGE  Architecture 


The  AEDGE  product  is  based 
on  an  extensible  distributed 
component-based  architecture,  which 
defines  entities,  agents,  roles,  and 
their  interactions.  The  interaction 
and  communications  among  AEDGE 
elements  are  based  on  the  Service 
Provider/Service  Requester  Protocol 
(SPSRP),  using  flexible  services  and 
messages  to  exchange  information  between  any  two  components  (Bames,  Petrov,  Elliott, 
&  Stoyen,  2002). 

In  SPSRP,  Service  Providers  implement  a  number  of  services  and  register  service- 
templates  with  a  Component  Registry,  which  maintains  the  location  of  all  components 
and  the  services  provided  or  required  by  each.  The  registry  is  used  by  Service  Requesters 
to  locate  components  that  provide  the  services  required  by  the  requester.  After  the 
requester  is  matched  with  one  or  more  Service  Providers,  a  direct  connection  is 
established  between  Service  Providers  and  Service  Requesters.  This  prevents  the  Registry 
from  being  a  service  dispatcher  and  a  potential  bottleneck. 

Let  us  consider  the  interaction  between  users  (via  user  interface  components)  and 
agent  components.  In  most  cases  users  will  interface  with  specialized  Agents,  called 
Agent  Managers.  The  managers  are  designed  to  coordinate,  synchronize  and  manage  the 
work  of  multiple  “worker”  agents.  Without  an  Agent  Manager,  the  user  will  need  to 
interface  with  each  “worker”  agent  individually.  Using  the  Manager,  an  individual  is 
able  to  issue  higher  level  requests  (e.g.,  “Send  me  your  current  recommendations”)  by 
letting  the  Agent  Manager  (who  knows  the  capabilities  of  its  workers)  to  distribute  and 
correlate  individual  agent  tasks.  The  Service  Requester  then  sends  Service  objects 
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directly  to  one  or  more  Service  Providers,  who  respond  with  Service  Result  objects.  A 
Requester  may  wish  to  subscribe  for  service  updates,  in  which  case  the  Service  Provider 
will  send  a  Message  object  to  the  requester  every  time  it  needs  to  advertise  an  update;  it 
is  up  to  the  Requester  to  respond  to  that  message  by  requesting  the  actual  update  (i.e., 
data  is  advertised,  not  pushed,  to  avoid  client-side  congestion). 

In  Figure  3,  we  see  the  user 
may  trigger  a  request  for 
recommendations,  which  is  sent  to 
the  Agent  Manager  via  a  Service 
Request  object.  After  receiving  the 
request,  the  Agent  Manager  finds 
the  best-suited  collection  of  agents 
to  perform  the  job  (it  may  take  one 
or  more  worker  agents)  and 
forwards  specialized  Service 
Requests  to  each  of  them.  The 
worker  agents  formulate  their 
responses  (usually,  sets  of 
recommendations  and  rationale)  and  send  them  back  to  the  Agent  Manager  via  Service 
Result  objects.  The  Agent  Manager  may  correlate  (and  even  remove  redundant  or 
inconsistent  recommendations)  all  service  results  and  then  sends  the  combined  set  of 
recommendations  to  the  user,  again  via  a  Service  Result  object.  The  user  interface 
component  then  knows  how  to  extract  the  recommendations  and  present  them  to  the 
human  user  for  evaluation. 
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AEDGE  Decision  Algorithms 


The  AEDGE  architecture  provides  multiple  levels  of  agent-based  algorithms. 
Generic  resource  allocation,  search,  and  optimization  algorithms  are  a  core  part  of  the 
AEDGE  product.  Each  AEDGE  application  can  use  and  further  extend  these  fundamental 
agent  algorithms  by  either  providing  parameters  and  applications-specific  values, 

functions  and  rules,  or  by  combining, 
modifying  or  supplying  new 
algorithms.  All  new  and  modified 
algorithms  must  comply  with  a  well- 
defined  agent  interaction  interfaces, 
similarly  to  the  generic  algorithms. 

The  AWACS-AEDGE  extends 
resource  allocation,  optimization,  and 
other  algorithms  with  A  WAGS  WD- 
specific  objective  functions  and 
constraints.  For  example,  the  AWACS  weapon-target  allocation  algorithm,  based  on  a 
generic  resource-allocation  with  heuristic  function  evaluation,  defines  extended 
constraints  such  as  Table  1. 

Similarly,  the  AWACS  weapon-target  allocation  algorithms  define  objective  and 
cost  functions  for  any  potential  allocation  and  let  the  generic  allocator  agent  arrive  at  a 
near-optimal  set  of  weapon-target  pairings.  The  objective  functions  are  based  on  the 
individual  target  values  (as  well  as  other  factors,  such  as  target  priorities,  probabilities  for 
success  and  so  forth).  Cost  functions  are  based  on  the  risk  for  the  team  if  the  allocation  is 
to  be  committed. 

Further,  the  AWACS-AEDGE  agents  use  the  extended  algorithms  as  a  model  of 
the  desired  WD  perfonnance.  Thus,  the  agents  are  able  to  generate  a  set  of 
recommendations  pertinent  to  a  particular  tactical  position  and  the  events  that  lead  to  it. 
Such  recommendations  can  either  be  presented  to  the  user  (who  may  choose  to  accept  or 
ignore  them)  or  be  used  for  internal  evaluation  of  the  user’s  perfonnance  as  a  function  of 
the  similarity  of  recommended-action  versus  actually-executed-action.  A  new  application 


Table  1:  Sample  Decision  Making  Algorithms 


IN  RANGE  (TargetCandidate, 
WeaponCandidate)  OR 

INTERCEPT  TIME  (WeaponPlatform, 
TargetCandidate)  <  MAX  TIME 
AND 

Pk  (TargetCandidate,  WeaponCandidate) 
>  DesiredPk  (TargetCandidate) 

AND 

FUELT  OINTERCEPT 
(WeaponPlatform,  TargetCandidate)  + 

FUELTOBASE  (InterceptPoint, 
WeaponPlatform)  <  WeaponPlatform.currentFuel 
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of  the  agent  recommendation  analysis  involves  the  measurement  of  AWACS  WD 
cognitive  workload  based  on  the  volume  and  complexity  of  agent  recommendations  at 
any  given  time  (Chaiken  et  ah,  2001). 

To  enhance  the  utility  of  AWACS-AEDGE  as  a  decision  support  tool, 
recommendations  must  not  only  be  presented  to  the  user  in  a  unambiguous  and  intuitive 
manner,  but  in  some  cases  they  may  need  to  be  pre-processed  to  ensure  that  the  human 
user  can  maintain  strong  situational  awareness  and  be  alert.  For  example,  in  periods  of 
exceptionally  high-activity,  unsupervised  agents  may  tend  to  generate  large  number  of 
recommendations  that  will  be  confusing  and  even  detrimental  to  the  human  performance. 
The  Agent  Manager  must  thus  not  only  coordinate  and  synchronize  recommendations, 
but  also  prioritize  and  reduce  the  number  of  presented  recommendations  to  only  the  top¬ 
most  critical  ones  (Chaiken  et  ah,  2001).  Conversely,  in  periods  of  a  lull,  artificially 
increasing  the  number  of  recommendations  may  help  keep  the  human  alert  and 
situationally  aware. 

AGENT  TECHNOLOGY 

Intelligent  agent  technology  is  rapidly  demonstrating  its  value  to  operational 
simulation  and  training.  Within  the  AEDGE,  "agent"  is  a  broadly  defined  term  with  three 
dominant  functions.  The  first  involves  the  simulation  of  scenario  entities,  referred  to  as 
constructed  forces.  Related  to  that  is  the  use  of  agents  to  substitute  for  other  human  roles, 
to  create  “synthetic”  team  members.  The  third  function  is  that  of  decision  support. 

Constructive  forces.  Agent  technology  defines  the  operating  characteristics  and 
behavior  of  hostile  and  friendly  entities  (e.g.  speed,  radar  range,  and  weapons  range  of 
aircraft).  Agent  specification  is  complete  and  detailed  to  the  point  where  the  entire 
scenario  can  be  played  out  through  a  federation  of  numerous  "agents"  (i.e.,  a  simulation 
with  no  live  players). 

Synthetic  team  members.  Agent  technology  was  also  applied  to  simulate  other 
WD  team  members  within  the  AEDGE.  The  distinction  between  "agent  as  constructive 
forces"  and  "agent  as  player"  is  largely  one  of  degree.  However,  the  latter  sort  of  agent  is 
typically  far  more  complex,  as  these  agents  are  designed  to  simulate  another  simulation 
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player,  not  just  a  battle  entity.  Such  agents  give  users  the  option  to  play  with  other  “live” 
participants  or  participate  alone,  with  the  simulation  acting  out  other  roles  in  a  realistic 
fashion.  This  sort  of  agent  also  defines  the  pedagogical  goal  of  the  simulation,  in  the 
sense  that  these  agents  can  be  used  to  implement  (e.g.,  set  policy  for)  optimal 
performance.  They  can  also  be  used  to  demonstrate  the  results  of  flawed  perfonnance. 
"Player"  agents  are  an  extremely  useful  and  yet  rare  capability  for  team  task  simulations. 
However,  their  development  is  expected  to  increase  given  the  great  utility  of  allowing 
individual  training  within  a  team-like  context.  Both  entity  and  player  agent  technology 
are  equally  important  to  our  effort.  The  former  sense  is  what  gives  the  AWACS-AEDGE 
its  fidelity  to  the  real  task;  the  latter  sense  provides  both  the  model  of  normative  behavior 
the  user  should  strive  for  and  the  means  (algorithms/knowledge)  to  effectively 
accomplish  that  behavior. 

Decision  Support.  The  third  manner  in  which  agent  technology  was  utilized  is  to 
provide  decision  support.  This  type  of  agent  is  not  so  much  a  simulation  of  a  player  (or 
simulation  of  an  entity)  but  a  simulation  of  a  "coach"  or  "adviser."  This  variant  of  agent 
can  be  very  broad,  and  the  distinction  between  such  agents  and  operational  interfaces  can 
be  blurry  because  such  agents  manifest  themselves  through  the  interface.  For  instance, 
such  agents  can  be  imbued  with  the  capability  to:  (a)  seek  out  information  over 
distributed  networks,  (b)  search  through  information  databases,  (c)  manipulate 
information  through  filtering,  transforming,  aggregation,  and  fusing  of  multiple, 
independent  information  steams;  and  (d)  report  information  to  the  human  requester. 
There  may  also  be  multiple  agents  working  on  several  tasks  at  any  point  of  time,  e.g., 
several  agents  monitoring  and  filtering  information  from  disparate  channels,  agents  to 
aggregate  and  fuse  relevant  information,  agents  to  select  an  appropriate  visualization  of 
the  data  to  report,  and  so  on.  Some  agents  may  be  imbued  with  a  high  level  of  autonomy, 
allowing  them  to  make  critical  recommendations  based  on  information  found  without 
human  solicitation  or  guidance. 

In  the  AEDGE,  the  experimenter  can  control  the  autonomy  and  configuration  of 
the  decision  aide  agent.  If  the  agent  is  allowed  to  make  all  decisions,  the  scenario  is 
effectively  being  run  independently  of  any  human  intervention.  This  allows  (a) 
assessment  of  reliability  of  recommendations,  (b)  assessment  of  effects  of  uncertainty  in 
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a  dynamic  environment,  and  (c)  investigation  of  “what-if  ’  scenarios,  where  algorithms 
underlying  recommendations  are  manipulated. 


DMT-RNet:  CONNECTIVITY 

The  AWACS  DDD-Net  was  the  first  step  of  implementing  and  demonstrating  the 
feasibility  of  the  DMT-RNet  that  allowed  distributed  simulations  over  the  Internet  in  real¬ 
time  (Bames,  et  ah,  2001).  Aptima,  Inc.  worked  with  faculty  and  staff  at  the  University 
of  Central  Florida  (and  associated  Institute  for  Simulation  Technologies  [UCF/IST]),  the 
University  of  South  Florida,  and  researchers  located  at  Brooks  Air  Force  Base  (Brooks 
City  Base)  to  test  the  feasibility  of  the  project  (Entin,  Serfaty,  Elliott,  &  Schiflett,  2001). 
The  DDD-Net  linked  the  different  locations  (as  shown  in  Figure  4  labeled  DDD)  to  form 
a  simulated  C2  infrastructure  that  allowed  multi-role  missions,  data  collection,  and  web- 
based  feedback  while  running  a  complex  C2  scenario. 

Different  parts  of  the  network  included  Intemet-2  (1-2)  connections  for  improved 
speed,  performance,  and  reliability.  The  DDD-Net  achieved  and  maintained  a 
synchronized  connection  for  an  AWACS  simulation  involving  16  participants. 
Simultaneously,  observers  at  each  location  rated  perfonnance  using  web-based  tools  that 
allowed  immediate  data  pooling,  analysis,  and  feedback,  within  10  minutes  after  data 
input  was  complete. 

While  the  previous  effort  connected  several  nodes  located  at  various  sites,  the  nodes 
were  only  extensions  of  the  same  (DDD)  platform.  The  next  effort  of  this  project 
extended  the  interoperability  capabilities,  as  well  as  developed  additional  platforms 
(Bames,  Petrov,  et  al.,  2002).  The  AWACS  AEDGE™  (as  discussed  above)  was 
successfully  connected  to  the  Command,  Control,  and  Communications  Simulation 
Training  and  Research  System  (C3 STARS)  at  Brooks  AFB,  TX  to  simulate  the  weapons 
director  roles  within  the  AWACS  (see  Figure  4,  labeled  AEDGE).  This  entailed  one-way 
communication  from  the  C3 STARS  to  the  AEDGE.  Position,  heading,  altitude,  and 
identification  data  were  all  communicated,  and  all  information  was  accurate  with  the 
exception  of  some  problems  with  the  heading  information.  However,  this  test  was  not 
run  in  the  reverse  direction,  with  information  going  from  the  AEDGE  to  the  C3STARS, 
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although  it  is  believed  that  it  would  likely  work.  Another  potential  advance  would 
involve  utilizing  the  AEDGE  agent  recommendation  infonnation  and  sending  it  to  the 
C3 STARS  operator. 


An  UAV  platfonn  developed  by  Cooke  and  Shope  (see  this  volume  for  a  detailed 
description)  was  connected  at  New  Mexico  State  University  to  a  cadre  of  fighter-pilot 
simulators  using  the  F-16  4-ship  Flight  Performance  Assessment  Simulation  System  (F- 
PASS)  developed  by  NOVA  Technologies,  Inc.  As  shown  in  Figure  4,  a  successful  test 
was  run  linking  the  C3STARS,  the  UAV,  and  the  FPASS  simulators  (Bames,  Petrov,  et 
al.,  2002).  The  C3STARS  and  UAV  simulators  utilized  T1  lines,  while  the  FPASS  used 
a  56K  baud  line.  While  connected,  all  three  of  the  systems  participated  in  a  “test 
exercise.”  The  FPASS  accurately  displayed  both  the  UAV  and  the  C3STARS-generated 
entities.  The  C3STARS  displayed  and  tracked  all  entities  involved. 

As  an  example  of  how  these  varied  simulators  were  integrated,  the  UAV  found 
ground  threats  and  this  data  was  sent  to  the  C3STARS.  The  C3STARS  then  used  these 
data  to  direct  friendly  forces,  including  both  FPASS  and  C3STARS-generated  entities. 

However,  one  limitation  this  test  faced  was  the  UAV  was  set  to  transmit  data  only.  It 
did  receive  data  from  the  C3STARS,  but  there  was  no  way  for  it  to  utilize  the  data.  Also, 
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voice  communications  were  sent  over  phone  lines.  In  the  future,  voice  communication 
will  be  integrated  in  the  network. 

TECHNICAL  ISSUES 

There  were  many  technical  issues  that  came  up  while  the  DMT-RNet  was  being  set 
up  (Barnes,  et  al.„  2001).  Different  parts  of  the  network  were  connected  at  different 
speeds.  Brooks  Air  Force  Base  used  a  T3  connection,  while  other  sites  had  only  T1 
connections.  Some  of  the  network  used  part  of  the  backbone  of  Abilene,  one  of  the  main 
1-2  branches.  Even  with  the  lowest  speed  being  a  T1  connection,  bandwidth  was  an 
issue,  mainly  because  of  the  mismatched  speeds.  To  make  things  more  complicated,  not 
only  was  1-2  employed,  but  the  secure  Defense  Research  and  Education  Network 
(DREN)  was  utilized  for  part  of  the  network. 

Getting  all  of  these  different  nodes,  firewalls,  and  networks  to  link  together  was  at 
times  a  complex  undertaking.  Incoming  and  outgoing  security  was  an  issue;  especially 
considering  the  fact  that  one  of  the  nodes  was  at  Brooks  Air  Force  Base.  There  were 
firewalls  to  be  dealt  with  that  were  necessary  for  security,  but  they  hindered  the  speed  of 
the  network  and  made  the  operation  more  complex.  Brooks  in  particular  had  not  only 
firewalls,  but  also  internal  and  external  routers  and  a  proxy  server  to  be  dealt  with  as 
well. 

Technical  issues  became  more  complex  as  the  multiple  simulators  were  being 
integrated.  Developed  individually,  these  simulators  were  not  originally  intended  to  be 
linked  with  other  types  of  simulators  in  a  DMT-RNet  type  of  environment.  Solving  these 
interoperability  issues  during  the  development  of  the  platforms,  and  having  them  work 
together  in  a  way  that  enables  this  type  of  research  was  one  of  the  major 
accomplishments  of  the  DMT-RNet. 

Currently,  the  DMT-RNet  uses  unclassified  information.  However,  being  connected 
on  a  military  base  raises  network  security  issues  that  are  important.  Base  networks  are 
under  constant  attacks  from  people  trying  to  hack  in,  consequently  there  are  stringent 
firewalls  that  the  DMT-RNet  has  to  navigate.  In  the  future,  security  issues  will  be  at  least 
as  complex  as  they  currently  are.  Different  research  networks  will  be  formed  and 
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communication  between  them  may  be  an  issue,  but  1-2  technology  will  form  a  stable  base 
that  will  make  that  a  more  possible  venture. 

DISCUSSION 

We  expect  the  platforms  within  DMT-RNet  to  enhance  research,  training,  and 
performance  in  complex  high- tempo  scenarios.  The  benefits  of  this  general  approach  to 
STE-based  research  is  detailed  elsewhere  (Schiflett  &  Elliott,  2000).  To  summarize 
briefly,  the  AWACS  AEDGE™  was  developed  to  primarily  to  support  trainers  and 
researchers.  In  fact,  every  characteristic  and  feature  within  this  platform  was  developed 
to  empower  trainers  and  researchers  with  regard  to  methods,  measures,  manipulations, 
and  transfer  of  training.  First,  internal  validity  is  enhanced  by  providing  researchers  with 
more  detailed  performance  measures,  increased  scenario  realism,  ease  in  generating 
scenario  events,  agent-based  performance  models,  and  comprehensive  data  output  files. 
In  addition,  further  control  is  provided  to  team  performance  researchers  through  the 
provision  of  synthetic  team  members — thus  allowing  investigations  of  performance 
within  more  highly  controlled  team  contexts.  It  provides  trainers  with  online  scenario 
revision  capabilities  and  visual  online  perfonnance  feedback  for  operators.  Finally,  this 
system  was  developed  to  enhance  external  validity — the  degree  to  which  research 
transitions  to  the  operational  perfonnance  environment.  This  was  accomplished  through 
comprehensive  cognitive  task  analyses  of  the  operational  performance  domain.  While 
use  of  this  system  is  no  guarantee  of  good  training  or  research  per  se,  we  hope  it  will 
accomplish  its  purpose — to  provide  tools  that  empower  experts  to  more  easily 
accomplish  research  and/or  training  goals. 

These  efforts  toward  the  DMT-RNet  have  shown  the  great  potential  that  exists  for  the 
integration  of  simulators  into  a  common  network,  one  that  can  be  utilized  not  only  for 
training  purposes  but  also  for  training  research.  Each  of  the  DDD,  AEDGE,  UAV, 
FPASS,  and  C3STARS  platforms  were  developed  independently,  without  plans  for 
integration.  However,  tests  have  been  successfully  run  showing  them  working  together, 
generating  and  displaying  entities  from  each  of  the  various  systems  in  a  common 
battlespace.  Integration  barriers  are  being  taken  down,  allowing  for  interoperability 
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capabilities  previously  undreamed  of.  This  will  allow  for  high-quality  collaborative 
research,  where  the  strengths  of  each  simulator  and  platform  are  put  to  use  in  a 
complementary  way.  The  creation  of  this  infrastructure  is  more  than  a  demonstration  of 
technology.  First,  the  platfonns  themselves  offer  unique  capabilities,  regarding  scenario 
realism,  experimental  control,  and  perfonnance  measures  to  experimenters.  The  use  of  a 
common  platform  allows  comparison  of  diverse  programs  of  research,  each  focused  on  a 
different  aspect  of  performance,  albeit  training,  interface  technology,  information 
distribution,  or  fatigue  countermeasures. 

Even  so,  why  connect  the  platforms?  Certainly,  the  distribution  and  portability  of 
these  platforms  have  self-evident  benefits  for  trainers — training  can  occur  among 
distributed  trainees,  deployed  in  remote  sites.  But  what  is  the  benefit  for  researchers? 
First,  there  is  great  importance  in  perfonning  research  on  multi-team  system 
performance.  Operational  teams  are  often  assigned  ad  hoc,  with  team  members  and 
teams  having  diverse  perspectives  and  little  familiarity  with  each  other.  Distributed  team 
research  can  capture  these  inherent  differences  faced  by  DMT  teams.  Teams  in  different 
locations  would  have  diverse  individuals,  trained  in  diverse  locations,  with  different 
curricula,  by  different  trainers — thus  capturing  relevant  and  realistic  diversity  in 
operations. 

In  addition,  operational  DMT  generates  questions  regarding  the  type  and  nature  of 
joint  mission  planning,  multi-team  coordination,  and  joint  debriefing  procedures.  DMT 
teams  encompass  diverse  teams,  such  as  cadres  of  fighter  pilots,  cargo  and  refueling 
aircrew,  and  various  C2  platforms.  DMT  itself  does  not  lend  itself  as  well  to 
experimental  design.  DMT  resources  are  devoted  more  toward  training  in  itself,  subjects 
/  resources  are  difficult  to  procure,  and  experimental  manipulation  more  difficult  to 
achieve  in  a  context  rich  in  confounding  variables  and  low  in  statistical  power.  The 
connection  of  diverse  internet-based  platforms  allow  university-based  researchers  to 
investigate  questions  of  skill  acquisition  in  multi-team  context,  to  easily  manipulate 
distribution  and  display  of  information  and  performance  feedback,  and  to  study  processes 
of  joint  mission  planning  and  debriefing.  Thus,  DMT-RNet  will  serve  as  a  scientific 
bridge  for  the  enhancement  of  operational  distributed  mission  training. 
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